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ABSTRACT 

A  method  for  directly  computing  iv.age  motion  at  corners 
is  presented.  The  method  is  based  on  temporal  intensity  gra¬ 
dients  along  lines  parallel  to  the  sides  of  the  corner.  The 
results  of  applying  the  method  to  two  time-varying  images  are 
discussed. 
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1.  Introduction 


The  earliest  problem  that  arises  in  the  analysis  of 
time-varying  images  is  the  detection  of  moving  image  ele¬ 
ments  Cedge,  regions)  and  the  computation  of  the  image 
velocity  Coptic  flow)  of  those  elements.  A  variety  of  com¬ 
putational  schemes  have  been  proposed  to  solve  this  problem. 

In  a  recent  survey,  Ullman  £1]  broadly  classifies  these  as 
intensity-based  and  token -matching  schemes. 

An  important  class  of  intensity-based  schemes  takes  ad¬ 
vantage  of  the  relationship  between  the  temporal  and  spatial 
gradient  of  any  continuous  and  differentiable  image  proper¬ 
ty  which  is  invariant  to  small  changes  in  perspective.  For  exam¬ 
ple,  if  we  assume  that  the  intensity,  I,  satisfies  these 
properties,  the  relationship 

-It  -  ulx  +  vly 

can  be  used  to  determine  velocity.  Here,  Ifc  is  the  temporal 

intensity  gradient,  I  and  I  the  x  and  y  components  of  the 

x  y 

spatial  intensity  gradient,  and  u  and  v  the  x  and  y  compo¬ 
nents  of  image  velocity.  Measuring  I. ,  I  ,  I  from  an  image 

r.  x  y 

sequence  establishes  a  linear  constraint  on  the  x  and  y  velo¬ 
city  components.  A  single  velocity  estimate  can  be  computed 
by  spatially  combining  the  constraints  using  e.g.,  Hough 
transforms  [2] ,  least-squares  methods  [3]  or  minimization 
techniques  [4] .  All  of  these  techniques  suffer  from  certain 
disadvantages.  The  Hough-transform  and  minimization  techniques 
assume  that  image  velocity  is  uniform  over  large  parts  of 


/ 


the  image,  and  the  least-squares  method  further  assumes  that  the 
constraint  equations  determined  for  nearby  points  are  in¬ 
dependent  -  an  assumption  that  is  violated  by  the  spatial 
integration  required  to  compute  spatial  derivatives.  An 
alternative  to  these  approaches  would  be  to  compute  mul¬ 
tiple  constraint  equations  of  a  single  point  based  on 
several  invariant  image  properties  -  e.g.,  intensity,  deriva¬ 
tives  of  intensity,  color.  This  approach  is  described  in 
Thrift  et  al.  [5] . 

This  is  the  first  in  a  series  of  reports  which  describes 
an  approach  to  image  velocity  estimation  and  subsequent  mov¬ 
ing  object  extraction  and  tracking.  Although  the  approach 
utilizes  some  of  the  gradient-based  motion  estimation  tech¬ 
niques  mentioned  above,  it  has  significant  differences  from 
those  techniques.  - ^  /U-y^  ^ 

First,  we  initially  compute  the  motion  estimates  at 
only  a  small  subset  of  image  points.  These  are  points  at 
which  it  is  possible,  in  principle,  to  determine  image  velo¬ 
city  with  little  or  no  integration  of  spatial  information. 

In  this  way  we  avoid  making  any  unnecessary  assumptions  about 
even  the  local  distribution  of  image  velocities.  Furthermore 
the  actual  estimated  velocities  are  subject  to  certain 
statistical  and  heuristic  confidence  tests,  which  can  further 
reduce  the  initial  set  of  points  to  which  motion  vectors 


are  attributed.  This  paper  is  concerned  with  this  first 
step  and  describes  techniques  for  measuring  image  motion 
at  comers.  Corners  have  the  property  that  their  motion 
can  be  directly  computed  based  only  on  measurements  made 
at  the  corner,  ^(in  practice,  of  course,  one  must  examine 
a  small  neighborhood  of  the  cornerT^  Another  important 
property  of  corners  is  that  they  can  be  safely  regarded 
as  projections  of  scene  features  whose  general  appearance 
is  invariant  to  rigid  motion  -  e.g.,  an  image  corner  may 
be  the  projection  of  the  vertex  of  a  polyhedron,  or  of  a 
curvature  discontinuity  on  the  boundary  of  a  surface  mark¬ 
ing.^  Thrift  et  al. [5]  describes  a  complementary  approach 
in  which  several  image  surface  attributes  are  combined  at 
a  single  point  to  compute  a  motion  vector  at  that  point. 

They  also  describe  how  the  confidence  of  that  estimate  can 
be  evaluated. 

v  ^  The  second  step  involves  propagating  these  velocity  esti 
mates  to  a  larger  number  of  picture  points.  In  Wu  et  al 

A. 

[6]  we  will  describe  one  such  method  which  propagates 
velocity  vectors  along  image  contours.  The  principal  diffi¬ 
culties  with  developing  and  applying  such  techniques  are 
guaranteeing  that  the  propagation  technique  is  formally 
correct  (at  least  for  rigid,  planar  motions)  and  avoiding 
propagation  across  object  boundaries.  The  resulting  pattern 


of  velocity  vectors  may  be  loosely  regarded  as  a  "mosaic” 
of  Glass  patterns  [7] .  Davis  and  Narayanan  [8]  will  con¬ 
sider  the  problem  of  segmenting  such  Glass  pattern  mosaics. 
Subsequent  reports  will  consider  the  integration  of  motion 
information  across  many  frames. 
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2 .  Estimating  motion  at  corners 

The  motion  of  a  corner  can  be  computed  based  on  tem¬ 
poral  intensity  changes  along  lines  parallel  to  the  sides 
of  the  corner.  We  first  describe  velocity  computation 
for  the  case  of  translation  motion ,  and  then  consider 
translation  combined  with  rotation. 

Suppose  that  a  corner  simply  translates  from  point 
CQ  to  between  two  frames  tQ  and  t1(see  Figure  1). 

Let  O'  be  a  point  on  the  bisector  of  PCQR  and  let  O' A  and 
O'B  be  lines  parallel  to  CQP  and  CqR,  respectively  at 
some  unit  distance  from  CQP  and  CqR,  Suppose  that  |o'A|= 
|0'B|=l+m,  for  some  constant  m.  Finally,  assume  that  the 
intensity  inside  the  corner  is  1  and  outside  the  corner  is  0. 

Now,  at  time  tg,  the  average  intensity  along  line  seg¬ 
ments  O' A  and  O'B  is 

IO'A^t0^  “  Io'B^t0^  = 

If  Ax'  and  Ay'  are  the  components  of  the  translation  in 
the  directions  of  the  lines  O' A  and  O'B,  then 

I0'ACtl)  =  U+Ax')/(l+m) 

Vb^I1  =  tl+Ay')/(l+m> 

assuming  that  m  is  chosen  large  enough  so  that  max (Ax', Ay')  < 
m.  Finally,  Ax'  and  Ay'  can  be  computed  from 

AI0'A  “  I0* A ^fcl^  “  IO'A^t0^  “  Ax'/(l+m) 

AI0'B  *  I0'B^tl^  “  IO'B^t0^  *  Ay'/d+m) 

Once  Ax'  and  Ay'  are  computed,  the  components  of  the 


velocity  in  the  original  image  coordinate  system  can  be 
recovered  easily: 


Axl  f  cosa  cosg]  FAx'1 

AyJ  [sina  sinfJj  (.Ay'J 

The  practical  success  of  this  technique  depends  on  our 
ability  to  compute  several  corner  parameters  accurately. 

These  parameters  are 

1.  corner  location  at  tg, 

2.  corner  shape  tangles  a  and  6) ,  and 

3.  corner  contrast  (assumed  here  to  be  1) 

Section  3  discusses  the  computation  of  these  parameters. 
Next,  we  extend  the  previous  simple  analysis  to  include  rota¬ 
tion  as  well  as  translation.  We  will  treat  this  case  as  a 
translation  from  CQ  to  followed  by  a  rotation  about 
through  a  clockwise  angle  y  (see  Figure  2) .  Since  translation 
and  rotation  are  specified  by  a  total  of  three  parameters, 
we  could  extend  the  above  analysis  using  only  a  third  line 
segment  parallel  to  either  O' A  or  O’B.  Instead,  we  consider 
two  pairs  of  parallel  line  segments,  and  compute  the  displace¬ 
ments  in  the  directions  O' A  and  O'B  rather  than  directly 
computing  the  angle  y. 

Let  Ax't»  Ay't  be  the  translational  components  of  the 
motion  in  the  O' A  and  O’B  directions,  and  Ax'r  and  Ay'r 
the  corresponding  rotational  components.  Then 

Ax’t  +  Ax'r  =  (1+m)  AIq,a 
Ay't  -  Ay'r  =  (1+m)  AIQlg 

From  Figure  3,  we  see  that 


(1) 

(2) 


*»  *.  %  *. 


Ax*  r 

l+Wt 


Ax V  -  AX", 


where  6  is  the  distance  between  the  parallel  line  segments 
O' A  and  CD.  Similarly 

Ay '  v  =  Ay 1  ^  -  Ay" -r  (4) 

1+Ax't  6 

A1  so 


Ax'r  -  Ax"r  = 

a+m)  IIo.Attl)-IcDttl)1 

(5) 

Ay'r  -  Ay"r  = 

a^)H0.B(t1)-iEF(t1)] 

(6) 

Substituting 

(5)  and  (6)  into  (3)  and  (4) 

and  simplify 

ing,  we  obtain 

4x'r  *  Cl4y't  “  C1 

(7) 

c2&x't  *  Ay'r  *  'c2 

(8) 

where 


C1  *  ^  [WV  *  ZCDltl^ 

C2  -  itrLl  Hsplh*  - 

Solving  for  Ax't  and  Ay'  we  obtain 

AX»  =  (1+m) (Alp* a  ~  CtAIq^r)  -  Ci  (l+Cg)  (9) 

1  1+C1C2 

Av*  -  (1+m^C2AlQ'A  ~  C1AI0'B)  ~  C2(Cl“1) 

* 

Substituting  (7)  and  C8)  into  (9)  and  CIO),  we  can  also 
compute  Ax’r  and  Ay'r,  which  gives  us  a  complete  description 


of  the  motion  of  the  corner 


3 .  Applications 

The  corner  motion  model  described  in  Section  2  has  been  applied 
to  two  image  sequences  containing  two  frames  each  (Figures  4-5). 

Corners  are  initially  detected  using  the  corner  detector 
described  in  Kitchen  and  Rosenfeld  [  9  ] .  Next,  a  small  window 
around  each  corner  is  analyzed  to  obtain  a  more  accurate  descrip¬ 
tion  of  the  corner.  Based  on  the  assumption  that  the  corner 
locally  contrasts  with  its  surround,  a  local  thresholding  proce¬ 
dure  (Milgram  [10] )  is  used  to  segment  the  window.  The  corner 
is  then  relocated  to  a  maximum  curvature  boundary  point  in  the 
thresholded  window.  The  slopes  of  the  line  segments  meeting  at 
the  corner  are  computed  using  a  one-dimensional  (slope)  Hough 
transform  procedure  (only  slope  need  be  computed  since  the  lines 
are  constrained  to  pass  through  the  corner  point. )  The  corners 
detected  by  this  procedure  are  marked  with  dark  crosses  in 
Figures  4a  and  5a. 

To  overcome  the  effects  of  various  sources  of  error  on  the 
motion  estimation,  several  quadruples  of  line  segments  are  used 
to  compute  estimates  of  Ax't,  Ay't,  Ax'r  and  Ay'r,  with  the 
final  motion  estimate  taken  as  the  average. 

The  results  are  displayed  in  Tables  1  and  2.  The  estimated 
motion  vectors  were  obatined  by  the  authors'  examination  of  digi¬ 
tal  enlargements  of  the  images. 


For  most  of  the  corners,  the  results  are  quite  satisfactory. 
The  error  in  corner  8  in  Table  1  is  due  to  a  hole  in  the  object 


Figure  4.  First  motion  sequence.  Detected 
corners  marked  with  dark  crosses 
in  frame  1. 


a  b 

Figure  5.  Second  motion  sequence.  Detected 
corners  marked  with  crosses  in 
frame  1 . 
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Table  1.  Notion  vectors  for  comers  in  Figure  4a 
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Table  2.  Notion  vectors  for  comers  in  Figure  5a 


4.  Conclusions 

We  have  shown  how  the  velocity  of  a  corner  can  be  reli¬ 
ably  estimated  based  only  on  very  simple  measurements  of  tem¬ 
poral  intensity  changes  in  the  neighborhood  of  the  corner. 

The  success  of  the  proposed  technique  depends  mostly  on  our 
ability  to  reliably  detect  and  describe  corners,  but  also  to 
a  great  extent  on  our  being  able  to  identify  a  sufficiently 
large  neighborhood  of  the  corner  which  contains  only  the  cor¬ 
ner  and  one  component  of  the  background,  and  no  other  moving 
objects.  In  cases  where  such  a  neighborhood  does  not  exist, 
the  method  gives  poor  results.  The  need  for  very  high  resolu 
tion  near  the  corners  for  motion  estimation  suggests  that  for 
reasons  of  efficiency  the  corner  motion  estimation  problem 
should  be  reconsidered  based  on  a  multiresolution  image  repre 
sentation.  The  uses  of  pyramids  for  motion  detection  in 
general,  and  corner  motion  detection  in  particular,  will  be 
discussed  in  a  subsequent  report. 
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